Active 3D Segmentation through Fixation of Previously Unseen Objects

We present a method for active object segmentation based on integration of several cues; image point positions, binocular disparities and pixel colours. It serves as a framework for generation of object hypotheses of previously unseen objects in natural indoor scenes. The appearance, 3D shape and size of objects are modelled in an iterative manner using an approximate Expectation-Maximisation (EM) method, that takes the dependencies between neighbouring pixel labels into consideration, unlike typical methods that assume neighbouring pixels to be independent. To better cope with situations when an object is hard to segregate from the surface it is placed on, possibly due to ambiguities in appearence, we propose a flat surface model as a complement to the two models typically used in figure-ground segmentation. A flat surface assumption is reasonable, given that most objects in indoor scenes are placed on flat, or at least locally flat, surfaces. We will show that even if no such physical plane exists in the scene, foreground segmentation succeeds anyway, since the flat surface model will become just another background model. We further let the segmentation evolve over time, this in order to provide more information and gradually improve segmentation and to facilitate tracking. Figure-ground segmentation is done using three different models, each described by a set of parameters; the foreground model θ f , the background model θb and that of the flat surface θs. Each pixel has an associated label li ∈ {l f , lb, ls}, depending on which component it belongs to. The model parameters θ = θ f ∪ θb ∪ θs and the labellings of all pixels l = {li} are unknown and estimated from the measurements m = {mi} at each pixel. With EM the maximum likelihood estimate of the model parameters θ is computed iteratively in two steps. In the first step (E-step) the conditional distribution w(l) = P(l|m,θ ′) is computed using the current estimate θ ′ and in the second step (M-step) a new estimate is found by maximising Q(θ |θ ′) = ∑l w(l) logP(m, l|θ). Unfortunately, since this summation is done over N3 different labellings, where N is the number of pixels, it quickly becomes prohibitly expensive. To make it computationally tractable we replace w(l) with the product of the conditional marginals for each unobserved label, w(li) = P(li|m,θ ′). Since a measurement mi depends only on its associated label li, the second step becomes a maximisation of